Diffusion Adaptation over Networks

Adaptive networks are well-suited to perform decentralized information processing and optimization tasks and to model various types of self-organized and complex behavior encountered in nature. Adaptive networks consist of a collection of agents with processing and learning abilities. The agents are linked together through a connection topology, and they cooperate with each other through local interactions to solve distributed optimization, estimation, and inference problems in real-time. The continuous diffusion of information across the network enables agents to adapt their performance in relation to streaming data and network conditions; it also results in improved adaptation and learning performance relative to non-cooperative agents. This article provides an overview of diffusion strategies for adaptation and learning over networks. The article is divided into several sections: 1. Motivation. 2. Mean-Square-Error Estimation. 3. Distributed Optimization via Diffusion Strategies. 4. Adaptive Diffusion Strategies. 5. Performance of Steepest-Descent Diffusion Strategies. 6. Performance of Adaptive Diffusion Strategies. 7. Comparing the Performance of Cooperative Strategies. 8. Selecting the Combination Weights. 9. Diffusion with Noisy Information Exchanges. 10. Extensions and Further Considerations. 11. Appendix A: Properties of Kronecker Products. 12. Appendix B: Graph Laplacian and Network Connectivity. 13. Appendix C: Stochastic Matrices. 14. Appendix D: Block Maximum Norm. 15. Appendix E: Comparison with Consensus Strategies. 16. References.